Cloud Foundation (OpenStack platform )

Student Notes

Welcome to Week 8!

1. Case Study: The "Nebula Inc." Startup

To understand how a cloud is built, we will follow a continuous scenario for the next three weeks. You have been hired as the Lead Cloud Engineer for a new software startup called "Nebula Inc." Currently, they have no infrastructure—just a credit card and a dream. Your job is to build their Virtual Data Center (VDC) from scratch using OpenStackOpen-source cloud computing platform platform . This is not a theoretical exercise; you will be typing the actual commands that cloud administrators use daily to construct the digital fabric of the modern internet.

The Roadmap:

• Week 8 (Foundation): You must create the secure environment. This involves setting up the company's "TenantGrouping of users and resources (also called Project)" (Project), hiring the staff (Users), purchasing the software licenses (Images), and cabling the office (Networking).
• Week 9 (Compute): You will deploy their first web servers and secure them with firewalls.
• Week 10 (Storage): You will attach persistent storage for their customer database.

By the end of this module, you will have a fully functional, multi-tier cloud application environment running on infrastructure you defined yourself.

2. Deep Dive: Identity Management (Keystone platform identity/authentication service )

Keystone platform identity/authentication service is the central nervous system of OpenStack platform . If Keystone platform identity/authentication service is down, nothing works. It is responsible for Authentication (AuthNWho are you?") and Authorization (AuthZWhat can you do?"). In a physical building, Keystone platform identity/authentication service is the security guard at the front desk who checks your ID badge and decides which floors you are allowed to visit.

2.1 The Authentication Workflow (The "Token Dance")

When you run a command like openstack server list, a complex sequence of events, often called the "Token Dance," occurs in the background before you see any output.

Figure 1: The Keystone platform identity/authentication service "Token Dance" - Documenting the 7-step process of authentication and authorization

1. Credentials: Your client (CLI or Horizon platform web dashboard ) sends your Username, Password, and Domain to the Keystone platform identity/authentication service API.
2. Validation: Keystone platform identity/authentication service validates these credentials against its Backend (SQL or LDAP).
3. Token Issue: If the credentials are valid, Keystone platform identity/authentication service generates a Token (a temporary digital ID card) and sends it back to you.
4. Service Request: The client then sends the actual request to Nova platform compute service for VMs (GET /servers), placing the Token in the HTTP Header (X-Auth-Token) as proof of identity.
5. Token Validation: Nova platform compute service for VMs does not trust you or your token blindly. It authenticates the token by sending it back to Keystone platform identity/authentication service with the question: "Is this token valid?"
6. Confirmation: Keystone platform identity/authentication service validates the token's signature and expiry in its database. It replies to Nova platform compute service for VMs : "Yes, this is valid. The user is 'admin' and has the 'member' role."
7. Execution: Once confirmed, Nova platform compute service for VMs finally executes the command and returns the server list.

2.2 The Backend (Where are users stored?)

Keystone platform identity/authentication service is modular and capable of integrating with existing enterprise systems. It can store users locally or talk to external systems:

• SQL (Local): Users are stored in the OpenStack platform database (MariaDB). This is the standard configuration for small deployments and our lab environment.
• LDAP / Active Directory (Enterprise): In large enterprises, you do not want to create separate accounts for every system. Keystone platform identity/authentication service can plug directly into the corporate Active Directory. When a user logs in, Keystone platform identity/authentication service forwards the password to the Domain Controller for validation, ensuring Single Sign-On (SSO).

2.3 Token Providers (Fernet vs UUID)

The format of the token itself determines the performance of the cloud.

• UUID (Legacy): A random string stored in the Keystone platform identity/authentication service Database. The problem with this method is that every validation requires a Database Lookup. In massive clouds handling 10,000 requests per second, this would crush the database.
• Fernet (Modern): A cryptographic token format. The token contains the User ID and Expiry Time, encrypted using a secret key held by the Keystone platform identity/authentication service server. The major benefit is that Keystone platform identity/authentication service does not need to store the token in a database. To validate it, it simply decrypts the token significantly reducing database load and allowing Keystone platform identity/authentication service to scale infinitely.

2.4 The Hierarchy

• Domain: A high-level container (e.g., "Default" or "Corporate_A"). Use for multi-tenant isolation.
• Project (Tenant): The workspace. Resources (VMs, Networks) belong to a Project.
• User: The human or service account.
• Role: The permission set. A User must have a Role on a Project to do anything.

2.5 CLI Implementation (Case Study: Nebula Inc.)

Now we apply this theory to our startup. "Nebula Inc." requires a dedicated, isolated environment where its developers can work without interfering with other departments. In OpenStack platform , we achieve this "Multi-Tenancy" by creating a specific Project. This project will act as the container for all their future VMs, networks, and storage volumes. It also allows us to set quotas (e.g., "Maximum 10 CPUs") to control their budget.

Step 1: Create the Project

openstack project create --domain default --description "Nebula Inc. Production" nebula_prod

• Explanation:
  • --domain default: Specifies that this project lives in the default domain.
  • --description: Metadata for admins.
  • Result: Creates a record in the projects table. Returns a UUID (e.g., a1b2c3...).

Step 2: Create the User

openstack user create --domain default --password-prompt nebula_admin

• Explanation:
  • --password-prompt: Hides input for security.
  • Result: Creating a user identity does NOT grant access. The user is currently "unemployed".

Step 3: Assign the Role

openstack role add --project nebula_prod --user nebula_admin member

• Explanation:
  • member: The standard permission level (can create VMs/Networks but cannot delete other users).
  • Result: Creates a row in the role_assignments table linking User+Project+Role.

2.6 Identity Verification & Management

In a production cloud, security is an ongoing process, not a one-time setup. The "Principle of Least Privilege" dictates that we must continuously verify that only the correct people have access to our sensitive data. Simply listing the users in the system is insufficient; a user might exist but have no access to anything. To audit this, we must inspect the Role Assignments.

Auditing Access

openstack role assignment list --user nebula_admin --names

• Explanation:
  • --names: Resolves UUIDs to human-readable names.
  • Result: Displays exactly which project the user can access and with what level of authority.

Managing Users

• Disabling a User: bash openstack user set --disable nebula_admin

  • Explanation: Sets enabled=False in the DB. The user cannot request new tokens.

Reflection: 1. Why do we use tokens (like Fernet) instead of just passing the username/password to every service? (Hint: Performance and Security). 2. If you delete a user, their history disappears. If you disable them, it remains. Why is this important for audit trails?

Resources:

• OpenStack Keystone Guide
• AWS Identity and Access Management (IAM)
• Microsoft Entra ID (Azure AD)

3. Deep Dive: Image Management (Glance platform image service )

Every Virtual Machine needs a hard drive to boot from. In the physical world, you might walk around with a USB stick containing a Windows or Linux ISO installer. In the cloud, this is inefficient. Instead, we use Glance platform image service , the OpenStack platform Image Service. Glance platform image service acts as a central library where validated, pre-installed operating system templates are stored. When you launch a VM - , Nova platform compute service for VMs contacts Glance platform image service to request a copy of the "Master Image" to be streamed to the hypervisor.

3.1 Understanding Disk Formats

Not all virtual disks are created equal. You must choose the right format for your cloud workload:

RAW is a bit-for-bit copy of the disk. It offers the fastest performance because there is no overhead, but it is space-inefficient. A 10GB drive takes up 10GB of physical space, even if it is empty, making it slow to copy over the network.

QCOW2 - Type 1 hypervisor for virtualization Copy-On-Write disk image format (QEMU - Type 1 hypervisor for virtualization Copy On Write) is the standard format for OpenStack platform . It supports compression and "Thin Provisioning," meaning a 10GB drive with only 100MB of data takes up only 100MB of physical space. Crucially, it logic enables snapshot - capabilities, allowing you to save the state of a VM - instantly.

ISO is a read-only archive used for installation media. While essential for building images, it is rarely used in cloud "boot-from-image" scenarios because we prefer pre-installed operating systems.

3.2 Glance platform image service Architecture

Glance platform image service is split into distinct components to separate the metadata from the actual data payload.

Figure 2: Glance platform image service Architecture - The separation of the API, Registry (Metadata), and Backend Store (Data)

• Glance platform image service API: The front-end service that accepts user requests (e.g., "Upload this image", "List images"). It verifies the user's token with Keystone platform identity/authentication service before proceeding.
• Glance platform image service Registry: An internal service that stores the metadata about images (Name, Size, Format, Owner) in the SQL database.
• Backend Store: The driver responsible for storing the actual binary data (the heavy bits). While this can be a local file system (/var/lib/glance), production clouds typically use a distributed storage cluster like Ceph or an object store like AWS S3 to ensure data durability and accessibility across all compute nodes.

3.3 CLI Implementation (Case Study: Nebula Inc.)

For any software company, consistency is key. We cannot have one developer running Ubuntu 20.04 and another running Fedora 35, as this leads to the infamous "it works on my machine" problem. To solve this, Nebula Inc. enforces a Standard Operating Environment (SOE). We will upload a "Golden Image"—a pre-approved, security-hardened operating system template - image for quick deployment that all staff must use. For our initial testing phase, we will utilize CirrOS, a lightweight (15MB) Linux distribution designed specifically for validating OpenStack platform clouds, before graduating to full-sized Ubuntu Server images in production.

Step 1: Download the Source

wget http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img

• Explanation: We assume we are on the "Jumpbox" or Controller node. Browsers cannot upload directly to Glance platform image service CLI; the file must exist locally.

Step 2: Upload to Glance platform image service

openstack image create "nebula_standard_os" \
 --file cirros-0.5.1-x86_64-disk.img \
 --disk-format qcow2 \
 --container-format bare \
 --public \
 --min-ram 64 \
 --min-disk 1

• Explanation:
  • --disk-format qcow2: Defines how the bits are organized.
  • --container-format bare: Indicates no extra metadata wrapper (OVF) is around the file.
  • --public: IMPORTANT. By default, images are "Private" (only visible to the uploader). This flag makes it visible to all projects in the cloud (Nebula, Admin, Testing, etc.).
  • --min-ram 64: A metadata tag. Nova platform compute service for VMs checks this before booting. If a user tries to launch this on a Flavor - template - image for quick deployment defining vCPUs, RAM, and disk with 32MB RAM, Nova platform compute service for VMs will block the request to prevent a crash.
  • Result: The file is streamed into the Glance platform image service Backend Store, and a UUID is generated.

3.4 Managing Images (Day 2 Operations)

Once images are uploaded, they are not static. You may need to update their metadata or remove obsolete versions.

Listing Images

openstack image list

• Explanation: Returns a table of available images.
• Result: Checks ID, Name, and Status. Status should be active. If status is queued, the upload failed.

Updating Metadata (Properties) Sometimes we forget a flag or need to deprecate an OS.

openstack image set --property cpu_arch=x86_64 nebula_standard_os

• Explanation: Adds a custom key-value pair to the image metadata. The Scheduler use this to ensure the VM - lands on Intel hardware, not ARM.
• Result: The Glance platform image service Registry is updated; the actual file is untouched.

Deleting Images

openstack image delete nebula_standard_os

• Explanation: Marks the image for deletion.
• Result: The metadata is removed from the Registry, and the backend storage driver (Ceph/File) is instructed to free the space.

Reflection: 1. If you have a slow 1Gb link interconnecting your data centers, which image format (RAW or QCOW2 - Type 1 hypervisor for virtualization Copy-On-Write disk image format ) would be faster to replicate? Why? 2. Why is it dangerous to make every image --public? (Think about licensed software like Windows Server).

Resources:

• OpenStack Glance Guide
• AWS AMIs (Amazon Machine Images)
• Azure Compute Gallery

4. Deep Dive: Networking (Neutron platform networking service )

Neutron platform networking service is the "Software Defined Networking" (SDN) component of OpenStack platform . In a traditional data center, creating a new network segment means filing a ticket with the network team to configure VLANs on physical Cisco/Juniper switches. In OpenStack platform , Neutron platform networking service gives this power to the user. A tenant can create their own private switching infrastructure, subnets, and routers via API, without ever touching a physical cable.

4.1 What is SDN (Software Defined Networking)?

Network Engineers often rely on physical switches and routers to move traffic. In the cloud, we virtualize this entirely using Software Defined Networking (SDN). The core concept of SDN is the separation of the Control Plane (The Brain) from the Data Plane (The Muscle).

Figure 3: Neutron platform networking service SDN Architecture - The separation of the Logical Control Plane (API) from the Physical Data Plane (Open vSwitch)

The Control Plane (Neutron platform networking service API/Server) acts as the brain of the operation. When you execute a command to create a network or open port 80, you are communicating with the Control Plane. It calculates the necessary logic and updates the state of the cloud database, but it does not touch a single network packet itself.

The Data Plane (OVS Agent/L2 Agent) sits on every compute node and acts as the muscle. It receives instructions from the Control Plane via a message bus (RabbitMQ) and implements them by programming the local virtual switch. It is the actual software responsible for moving packets from your VM - to the physical network card.

4.2 The Virtual Switch: Open vSwitch (OVS)

In a physical rack, servers plug into a top-of-rack switch. In OpenStack platform , VMs plug into a virtual switch called Open vSwitch (OVS). Unlike a standard unmanaged switch that simply learns MAC addresses, OVS is a production-quality, multilayer virtual switch that uses Flow Tables. A Flow Table is a list of programmable rules that match specific packets (e.g., "If source IP is A and dest IP is B...") and applies specific actions (e.g., "...drop packet" or "...tag with VLAN 100"). Neutron platform networking service programs these flow tables dynamically to implement sophisticated features like Security Groups (Distributed Firewalls) and Virtual Routing.

4.3 Under the Hood: The Linux Connection

Everything you learned in Week 4 applies here. Neutron platform networking service uses standard Linux kernel features to build these structures:

• Isolation = Namespaces: When you create a Router or a DHCP server, Neutron platform networking service creates a Linux Network Namespace (ip netns). This allows Project A and Project B to both use "192.168.0.1" without conflict; they live in parallel, isolated universes.
• Cabling = Veth Pairs: When a VMplugs in" to the OVS Bridge, Neutron platform networking service creates a Virtual Ethernet (veth) pair. One end connects to the VM - 's interface (inside KVM - Type 1 hypervisor ), and the other connects to the OVS Bridge.

4.4 Flow of Traffic (North-South vs East-West)

Designing a cloud network requires understanding the two primary directions of traffic flow, as they traverse different paths through the infrastructure.

Figure 4: North-South vs East-West Traffic - Visualizing how traffic stays within the cloud versus how it exits to the internet

East-West Traffic refers to communication between VMs inside the same cloud environment (e.g., Web Server A talking to Database Server B). Ideally, this traffic should never leave the virtual infrastructure. It flows from the source VM - , through the local OVS Bridge, and is typically encapsulated in a tunnel protocol like VXLAN to cross the physical network before arriving at the destination compute node.

North-South Traffic refers to communication entering or leaving the cloud (e.g., a User accessing your Web Server from the Internet). This traffic must leave the virtual overlay network. It passes through the Neutron platform networking service Router (which lives inside a Network Namespace), undergoes SNAT (Source NAT) to mask its private IP, and exits via the external provider network gateway.

4.5 CLI Implementation (Case Study: Nebula Inc.)

Now that we understand the theory of pipelines and flows, it is time to build. Nebula Inc. requires a private, isolated network segment where their web servers can communicate safely. We will construct a topology consisting of a private Virtual Switch (nebula_net), an IP addressing scheme (nebula_subnet), and a Virtual Router (nebula_router) to connect to the outside world.

Step 1: Create the Switch (Network)

openstack network create nebula_net

• Explanation: Initializes the logical switch in the database (Control Plane).
• Result: A network UUID is created. OVS is not touched yet.

Step 2: Define Addressing (Subnet)

openstack subnet create --network nebula_net \
 --subnet-range 192.168.50.0/24 \
 --gateway 192.168.50.1 \
 --dns-nameserver 8.8.8.8 \
 nebula_subnet

• Explanation:
  • --network nebula_net: Attaches this IP logic to the switch.
  • Result: The DHCP Agent (Data Plane) spawns a dnsmasq process in a namespace to serve IPs.

Step 3: Build the Gateway (Router)

openstack router create nebula_router

• Explanation: Creates a virtual router instance.

Step 4: Wiring (Interface Attachment)

openstack router add subnet nebula_router nebula_subnet

• Explanation: This is the equivalent of plugging a patch cable from the Switch (nebula_subnet) into the Router's LAN port.
• Result: The L3 Agent creates a generic router namespace and assigns the gateway IP 192.168.50.1.

Step 5: Uplink (External Gateway)

openstack router set --external-gateway public nebula_router

• Explanation: Connects the Router's WAN port to the Provider Network (public).
• Result: Enables the router to route traffic to the internet (North-South flow).

4.6 Verification

Log in to Horizon platform web dashboard -> Network -> Network Topology. You should see the Nebula Router creating a bridge between the Blue (Private) line and the Red (Public) line.

Reflection: 1. Recall: In Week 4, we used ip netns exec. How does that relate to a Neutron platform networking service Router? 2. Flows: If you add a Security Group rule to allow SSH, what actually changes on the Compute Node? (Hint: Does Neutron platform networking service update the database or OVS flow tables?)

Resources:

• OpenStack Neutron Guide
• AWS VPC (Virtual Private Cloud)
• Azure Virtual Network (VNet)

6. Industry Comparison: The "Polyglot" Cloud Engineer

While this course uses OpenStack platform because it allows us to see "under the hood," the concepts you learned this week—Tenants, Golden Images, and SDN—are the exact same primitives used by the public cloud giants.

6.1 Concept Mapping

6.2 The "Standard Operating Environment" across Clouds

In Section 3, we discussed the "Golden Image." This strategy is universal.

• OpenStack platform : You use Packer to build a QCOW2 - Type 1 hypervisor for virtualization Copy-On-Write disk image format image and upload it to Glance platform image service .
• AWS: You use Packer to build an AMI and upload it to EC2.
• Azure: You use Packer to build a VHD and upload it to Azure Compute Gallery.
• The Conclusion: The tool (Packer) and the workflow (Build -> Validate -> Upload) are identical; only the target file format changes.

7. Summary

We have built the Foundation for Nebula Inc.: 1. Identity: A secure Project and User with assigned Roles. 2. Image: A validated OS template - image for quick deployment in Glance platform image service . 3. Network: A fully routed Layer 3 topology.

Next week, we Launch.

4. Lab Exercises

• Lab 1: Cloud Foundation
  • Goal: Introduction to Keystone platform identity/authentication service (Identity) and Glance platform image service (Image) services.